model inference

All posts tagged model inference by Linux Bash
  • Posted on
    Featured Image
    A practical, copy‑pasteable guide to running ChatGPT‑style LLMs locally on Linux with pure Bash: pick the easy Ollama route (one‑command install, auto‑managed models, CLI + HTTP API) or build llama.cpp for full control and GPU acceleration (CUDA/ROCm/SYCL). Covers apt/dnf/zypper setup, GGUF models, chat/server examples, a handy ai() wrapper and REPL, tuning (quantization, threads, context, GPU offload), and real‑world automation use cases.